Papers by Livio Baldini Soares
New Protocols and Negative Results for Textual Entailment Data Collection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language inference data has proven useful in benchmarking and as pretraining data for tasks requiring language understanding. |
| Approach: | They propose four alternative protocols to improve annotation quality and diversity . they use 8.5k-example training sets to compare different protocols . |
| Outcome: | The proposed protocols improve the ease of training and quality of the examples. |
Matching the Blanks: Distributional Similarity for Relation Learning (P19-1)
Copied to clipboard
| Challenge: | Efforts to build general purpose relation extractors that can model arbitrary relations are limited in their ability to generalize. |
| Approach: | They propose to build task-agnostic relation representations solely from entity-linked text to extend Harris’ distributional hypothesis to relations. |
| Outcome: | The proposed representations outperform previous methods on SemEval 2010 Task 8, KBP37, and TACRED even without using any of the task’s training data. |
From RAG to Riches: Retrieval Interlaced with Sequence Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | RICHES interleaves retrieval with sequence generation tasks . traditional approaches chain LLM generation with separate retrieval model . |
| Approach: | They propose a novel approach that interleaves retrieval with sequence generation tasks . they propose attributed evidence, multi-hop retrievals and interleave thoughts to plan on what to retrieve next . |
| Outcome: | The proposed approach can work with any Instruction-tuned model, without additional training. |
Entities as Experts: Sparse Memory Access with Entity Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Unlike previous attempts to integrate entity knowledge into sequence models, EaE’s entity representations are learned directly from text. |
| Approach: | They propose a model that can access distinct memories of entities mentioned in a piece of text and a new architecture that can do this. |
| Outcome: | The proposed model outperforms an encoder-generator Transformer model with 10x the parameters on a task that requires 10x more parameters to answer. |
Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students? (2022.tacl-1)
Copied to clipboard
Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C. Lipton, Graham Neubig, William W. Cohen
| Challenge: | Existing methods to explain predictions by highlighting salient features are often unstated. |
| Approach: | They propose a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. |
| Outcome: | The proposed framework allows principled, automatic, model-agnostic evaluation of attributions. |
Adaptable and Interpretable Neural MemoryOver Symbolic Knowledge (2021.naacl-main)
Copied to clipboard
| Challenge: | Past research has shown that large neural language models encode surprising amounts of factual information, but augmenting or modifying this information requires modifying a corpus and retraining, which is computationally expensive. |
| Approach: | They propose a neural LM that includes an interpretable neuro-symbolic KB in the form of a "fact memory" their LM improves performance on knowledge-intensive question-answering tasks, sometimes dramatically . |
| Outcome: | The proposed model improves on knowledge-intensive question-answering tasks, including a 27 point increase in one setting of WebQuestionsSP over a state-of-the-art open-book model. |